High availability for cloud-based services

ABSTRACT

High availability mechanisms are described for cloud-based services such as voice-over-IP (VoIP) services.

RELATED APPLICATION DATA

The present application is a divisional of and claims priority under 35U.S.C. 120 to U.S. patent application Ser. No. 13/629,469 for HighAvailability for Cloud-Based Services filed on Sep. 27, 2012, the entiredisclosure of which is incorporated herein by reference for allpurposes.

BACKGROUND

The present invention relates to systems and methods for providingcloud-based services and, in particular, to techniques for ensuring highavailability for such services.

The term “cloud computing” refers to computing models for enablingubiquitous, convenient, on-demand network access to a shared pool ofconfigurable computing resources (e.g., networks, servers, storage,applications, and services). Cloud-based services are rapidly becomingthe primary way in which services are provided to businesses andconsumers over the Internet and the World Wide Web.

Users of cloud-based services generally expect such services to beavailable any time, in any geographic location, and using any of a widevariety of client devices. These expectations of “high availability”present a number of technical challenges for providers of cloud-basedservices.

SUMMARY OF THE INVENTION

According to the present invention, methods, apparatus, systems, andcomputer program products are provided for promoting high availabilityof services over a network, e.g., cloud-based services.

According to a particular class of implementations, methods, apparatus,systems, and computer program products are provided for provisioning aclient device to access a service provided by a services network havinga plurality of service end points associated with a plurality ofgeographically distinct data centers. In response to a request from theclient device to access the service, the client device is provisionedwith a plurality of domain name system (DNS) service records. Each of afirst subset of the DNS service records corresponds to a unicastdestination address. Each unicast destination address is associated withone or more of the service end points in only a single one of the datacenters. Each of a second subset of the DNS service records correspondsto an anycast destination address. Each anycast destination address issimultaneously associated with one or more of the service end points inat least two of the data centers.

According to some implementations, the first subset of DNS servicerecords is higher priority than the second subset of DNS servicerecords.

According to some implementations, the first subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to a first one of the data centers, and a second group ofone or more DNS service records corresponding to a second one of thedata centers. The first group of DNS service records is higher prioritythan the second group of DNS service records.

According to some implementations, the second subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to both of first and second ones of the data centers, anda second group of one or more DNS service records corresponding to bothof the first and second data centers. The first group of DNS servicerecords is higher priority than the second group of DNS service records.

According to another class of implementations, methods, apparatus,systems, and computer program products are provided for accessing aservice provided by a services network having a plurality of service endpoints associated with a plurality of geographically distinct datacenters. A first one of a plurality of domain name system (DNS) servicerecords is retrieved. Each of a first subset of the DNS service recordscorresponds to a unicast destination address. Each unicast destinationaddress is associated with one or more of the service end points in onlya single one of the data centers. Each of a second subset of the DNSservice records corresponds to an anycast destination address. Eachanycast destination address is simultaneously associated with one ormore of the service end points in at least two of the data centers. Afirst request to access the service is transmitted to the destinationaddress corresponding to the first DNS service record. Where the firstrequest to access the service is not successful, or where a connectionto the service established using the first DNS service record isinterrupted, a second one of the plurality of DNS service records isretrieved. A second request to access the service is transmitted to thedestination address corresponding to the second DNS service record.

According to some implementations, the first subset of DNS servicerecords is higher priority than the second subset of DNS servicerecords, and attempts to connect with the service are made using all ofthe first subset of DNS service records before an attempt to connectwith the service is made using any of the second subset of DNS servicerecords.

According to some implementations, the first subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to a first one of the data centers, and a second group ofone or more DNS service records corresponding to a second one of thedata centers. The first group of DNS service records is higher prioritythan the second group of DNS service records. Attempts to connect withthe service are made using all of the first group of DNS service recordsbefore an attempt to connect with the service is made using any of thesecond group of DNS service records.

According to some implementations, the second subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to both of first and second ones of the data centers, anda second group of one or more DNS service records corresponding to bothof the first and second data centers. The first group of DNS servicerecords is higher priority than the second group of DNS service records,and attempts to connect with the service are made using all of the firstgroup of DNS service records before an attempt to connect with theservice is made using any of the second group of DNS service records.

According to another class of implementations, at least onenon-transitory computer-readable medium has a data structure storedtherein for enabling a client device to access a service provided by aservices network having a plurality of service end points associatedwith a plurality of geographically distinct data centers. The datastructure includes a plurality of domain name system (DNS) servicerecords. Each of a first subset of the DNS service records correspondsto a unicast destination address. Each unicast destination address isassociated with one or more of the service end points in only a singleone of the data centers. Each of a second subset of the DNS servicerecords corresponds to an anycast destination address. Each anycastdestination address is simultaneously associated with one or more of theservice end points in at least two of the data centers. Each of the DNSservice records is configured to enable a client device to access theservice using the corresponding destination address.

According to some implementations, the first subset of DNS servicerecords is higher priority than the second subset of DNS servicerecords.

According to some implementations, the first subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to a first one of the data centers, and a second group ofone or more DNS service records corresponding to a second one of thedata centers. The first group of DNS service records is higher prioritythan the second group of DNS service records.

According to some implementations, the second subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to both of first and second ones of the data centers, anda second group of one or more DNS service records corresponding to bothof the first and second data centers. The first group of DNS servicerecords is higher priority than the second group of DNS service records.

According to another class of implementations, a services network isprovided that includes a plurality of computing devices deployed in aplurality of data centers. The plurality of computing devices isconfigured to provide a plurality of service end points associated witheach of the data centers. Each of the service end points is configuredto provide access to a service to requesting client devices. One or moreof the computing devices in each data center is further configured toassociate a range of unicast destination addresses and a range ofanycast destination addresses with the corresponding data center. Eachrange of unicast destination addresses is associated with only one ofthe data centers. Each range of anycast destination addresses isassociated with two or more of the data centers. The plurality ofcomputing devices are configured to provision each of first ones of therequesting clients to access the service with a plurality of domain namesystem (DNS) service records. Each of a first subset of the DNS servicerecords corresponds to a unicast destination address in one of theranges of unicast destination addresses. Each unicast destinationaddress is associated with one or more of the service end points in thecorresponding data center. Each of a second subset of the DNS servicerecords corresponds to an anycast destination address in one of theanycast destination ranges. Each anycast destination address issimultaneously associated with one or more of the service end points inat least two of the data centers.

According to some implementations, a first one of the anycastdestination addresses is associated with first and second ones of thedata centers, and traffic corresponding to the first anycast destinationaddress is routed to the second data center within the services network.According to some implementations, the traffic corresponding to thefirst anycast destination address is handled locally where the seconddata center is unable to handle the traffic.

According to some implementations, a first one of the anycastdestination addresses is associated with first and second ones of thedata centers, and both the first and second data centers handle trafficcorresponding to the first anycast destination address locally.

According to some implementations, each of second ones of the requestingclient devices is provisioned using only a single one of the DNS servicerecords from the second subset of DNS service records.

According to some implementations, the plurality of DNS service recordswith which each of the first requesting client devices is provisionedincludes two or more DNS service records from the first subset of DNSservice records, each having a priority associated therewith. Thepriorities of the two or more DNS service records correspond to a loadbalancing scheme.

According to some implementations, the first subset of DNS servicerecords is higher priority than the second subset of DNS servicerecords.

According to some implementations, the first subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to a first one of the data centers, and a second group ofone or more DNS service records corresponding to a second one of thedata centers. The first group of DNS service records is higher prioritythan the second group of DNS service records.

According to some implementations, the second subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to both of first and second ones of the data centers, anda second group of one or more DNS service records corresponding to bothof the first and second data centers. The first group of DNS servicerecords is higher priority than the second group of DNS service records.

According to some implementations, the service comprises a voice-over-IPservice.

According to another class of implementations, a client device isconfigured to access a service provided by a services network having aplurality of service end points associated with a plurality ofgeographically distinct data centers. The client device includes one ormore processors and memory configured to access the service using aplurality of domain name system (DNS) service records. Each of a firstsubset of the DNS service records corresponds to a unicast destinationaddress. Each unicast destination address is associated with one or moreof the service end points in only a single one of the data centers. Eachof a second subset of the DNS service records corresponds to an anycastdestination address. Each anycast destination address is simultaneouslyassociated with one or more of the service end points in at least two ofthe data centers. Each of the DNS service records is configured toenable a client device to access the service using the correspondingdestination address.

According to some implementations, the first subset of DNS servicerecords is higher priority than the second subset of DNS servicerecords, and the client device is configured to attempt to connect withthe service using all of the first subset of DNS service records beforeattempting to connect with the service using any of the second subset ofDNS service records.

According to some implementations, the first subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to a first one of the data centers, and a second group ofone or more DNS service records corresponding to a second one of thedata centers. The first group of DNS service records is higher prioritythan the second group of DNS service records, and the client device isconfigured to attempt to connect with the service using all of the firstgroup of DNS service records before attempting to connect with theservice using any of the second group of DNS service records.

According to some implementations, the second subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to both of first and second ones of the data centers, anda second group of one or more DNS service records corresponding to bothof the first and second data centers. The first group of DNS servicerecords is higher priority than the second group of DNS service records,and the client device is configured to attempt to connect with theservice using all of the first group of DNS service records beforeattempting to connect with the service using any of the second group ofDNS service records.

According to some implementations, the client device may be any one of acell phone, a smart phone, a tablet, a laptop computer, a desktopcomputer, a telephone, an Internet Protocol phone, a set top box, or agaming console.

According to another class of implementations, methods, apparatus,systems, and computer program products are provided for providing acommunication service. A plurality of service end points associated witheach of a plurality of geographically distinct data centers is provided.Each of the service end points is configured to provide access to thecommunication service to requesting client devices. A range of anycastdestination addresses is simultaneously associated with two or more ofthe data centers. Requesting client devices are provisioned to accessthe communication service using anycast destination addresses in theanycast destination range. In response to requests from the requestingclient devices using the anycast destination addresses, thecommunication service is provided to the requesting client devices withthe service end points associated with at least one of the two or moredata centers.

According to some implementations, a first one of the anycastdestination addresses is associated with first and second ones of thedata centers. Traffic corresponding to the first anycast destinationaddress is routed from the first data center to the second data centerwithin the services network. According to some implementations, thetraffic corresponding to the first anycast destination address ishandled locally at the first data center where the second data center isunable to handle the traffic.

According to some implementations, a first one of the anycastdestination addresses is associated with first and second ones of thedata centers, and traffic corresponding to the first anycast destinationaddress is handled locally in both the first and second data centers.

A further understanding of the nature and advantages of the presentinvention may be realized by reference to the remaining portions of thespecification and the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1 and 2 are simplified diagrams of an example of a communicationsystem with which various embodiments described herein may beimplemented.

FIG. 3 is a simplified diagram of a services system illustrating highavailability mechanisms.

FIG. 4 shows examples of DNS SRV records.

FIG. 5 illustrates an example of a flow in which a client deviceregisters with a service.

FIG. 6 is another simplified diagram of a services system illustratinghigh availability mechanisms.

FIG. 7 is yet another simplified diagram of a services systemillustrating high availability mechanisms.

FIG. 8 shows further examples of DNS SRV records.

DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS

Reference will now be made in detail to specific embodiments of theinvention including the best modes contemplated by the inventors forcarrying out the invention. Examples of these specific embodiments areillustrated in the accompanying drawings. While the invention isdescribed in conjunction with these specific embodiments, it will beunderstood that it is not intended to limit the invention to thedescribed embodiments. On the contrary, it is intended to coveralternatives, modifications, and equivalents as may be included withinthe spirit and scope of the invention as defined by the appended claims.In the following description, specific details are set forth in order toprovide a thorough understanding of the present invention. The presentinvention may be practiced without some or all of these specificdetails. In addition, well known features may not have been described indetail to avoid unnecessarily obscuring the invention.

Various implementations are described herein in the context cloud-basedcommunication services, e.g., telephony (e.g., VoIP) and fax services.However, it should be understood that the scope of the invention is muchbroader, encompassing a wide variety of systems and services providedusing virtually any network-based computing paradigm. To be clear, thetechniques described herein may be used to support any type of networkor cloud-based service or set of services. Therefore, references tospecific applications herein are not intended and should not be used tolimit the scope of the invention.

Various embodiments described herein may be implemented in or inconjunction with a cloud-based communication system such as the oneillustrated in FIGS. 1 and 2. FIG. 1 shows a communication system 100which can be, for example, a cloud-based telephony system that providesvoice and video over IP, fax services, etc. Communication system 100includes data centers 101, 102, and 103. Each data center is a point ofpresence (POP) that includes the network resources (e.g., servers,routers, switches, network connections, etc.) necessary to support theservices provided by communication system 100. Each data center istypically located in a different geographical region.

In this example, communication system 100 includes three user points ofdata (pods), i.e., pods 1, 2 and 3, each of which is a logical groupingof two or more pod units situated in different data centers. Each podserves a different subset of user accounts. In this example, each podunit (e.g., unit 2A) serves the same subset of users as the other podunits within the same pod (e.g., pod units 2B and 2C). Each pod unitincludes a communication server 119 a-119 g configured to providesubstantially the same services to the same subset of users as the otherpod units within the same pod. Each pod unit also includes an accountdatabase 121 a-121 g configured to support the respective communicationservers for the corresponding subset of users.

FIG. 2 shows various components of communication system 100 of FIG. 1.Specifically, FIG. 2 shows the various interconnections within andbetween data centers 101 and 102. Both data centers are in communicationwith network 217. Service requests from various communication devices243A-243D are routed through network 217 to either or both of the datacenters. Devices 243A-243D represent a great diversity of client devicesthat may connect with a services system designed in accordance with oneor more embodiments of the present invention. Such client devices mayinclude, for example (and without limitation), cell phones, smartphones, tablets, laptop and desktop computers, conventional telephones,IP phones, set top boxes, gaming consoles, etc. Reference to specificclient device types should therefore not be used to limit the scope ofthe invention.

Data center 101 includes pod units 1A and 2A, a common database (CDB)207A, a message storage system (MSS) 211A, a router 213A, and a globaluser directory (GUD) 215A. Additional pod units (not shown) may also beincluded in data center 101. Data center 102 is similarly configured andincludes components that operate substantially the same as those in datacenter 101. Data centers 101 and 102 provide backup and redundancy toone another in the event of failure.

Communication servers 119 provide telecommunication services (e.g.,voice, video, email, and/or facsimile) to corresponding subsets ofusers. Each server 119 may also provide other services including, forexample, user account management and configuration, billing services,accounting services, etc. Each pod unit includes an account database 121to support the communication server(s) for that particular pod unit,storing configuration details and other information regarding eachuser's account.

Pod units 1A and 1B are in communication with one another so that thedata on their respective account databases are synchronized across datacenters. Data center 101 includes router 213A to receive an incomingservice request 231A from network 217. Router 213A parses the incomingservice request to identify or extract a user key and queries GUD 215Ato determine which pod is associated with the user key. Once theassociated pod has been identified router 213A routes the servicerequest to the pod unit in the data center associated with theidentified pod. If the pod unit associated with the identified pod isnot associated with data center 101, router 213A may route the servicerequest to another data center (e.g., data center 102 as indicated bythe arrow 241A).

Each pod unit of the data center 101 is also coupled to MSS 211A whichstores files for the users served by pod units 1A and 2A. These filesmay include, for example, messages (e.g., voicemails and facsimiles),user logs, system messages, system and user call prompts (e.g.,auto-attendant or user-recorded greetings), and other types ofcall-related or electronic messages. The contents of MSS 211A aresynchronized with other data centers (e.g., synchronized with MSS 211Bof data center 102).

Each pod unit in data center 101 is coupled to common database 207Awhich stores shared data for all of the pods, and stores consolidatedinformation from account databases 121. Common database 207A alsofacilitates changes to the pod databases. For example, common database207A may store data for applications that provide the services oncommunication servers 119. Different versions of the applications datamay be stored in common database 207A which allow changes and upgradesto communication servers 119 to be implemented efficiently andconveniently. Changes may be made to common database 207A and propagatedto pod units 1A and 2A. Common database 207A is synchronized across datacenters to other common databases (e.g., common database 207B of datacenter 102). Common database 207A, MSS 211A, router 213A, and GUD 215Aform a common layer of resources that are shared by all pod units indata center 101.

For more information regarding the nature of such a system with whichvarious embodiments described herein may be used, please refer to U.S.patent application Ser. No. 12/957,125 entitled User Partitioning in aCommunication System filed on Nov. 30, 2010, the entire disclosure ofwhich is incorporated herein by reference for all purposes.

As will be understood, the diversity of client devices (e.g., devices243A-243D) that providers of cloud-based services would like to be ableto support presents a number of challenges, not only with respect toprovisioning the various types of devices to facilitate access to theservices, but also with respect to ensuring high availability of theservices. Therefore, according to various implementations, a cloud-basedservices system (e.g., system 100 of FIGS. 1 and 2) may employ multipletechniques for ensuring high availability of services in the face of themany possible types of interruptions that may occur.

According to a particular implementation in which at least some of thedevices accessing a cloud-based service are equipped with advanced DNSresolving mechanisms, a cloud-based services system is configured toprovide high availability using appropriately configured DNS servicerecords.

The Domain Name System (DNS) is a hierarchical distributed naming systemfor computers, services, or any resource connected to the Internet or aprivate network. It associates various information with domain namesassigned to each of the participating entities. A Domain Name Serviceresolves queries for these names into IP addresses for the purpose oflocating computer services and devices worldwide. A DNS service record(SRV record) is a specification of data in the DNS defining thelocation, i.e. the hostname and port number, of servers for specifiedservices.

An SRV record has the form:

_service._proto.name TTL class SRV priority weight port target

in which “service” is the symbolic name of the desired service; “proto”is the transport protocol of the desired service (usually either TCP orUDP); “name” is the domain name for which the record is valid; “TTL” isthe standard DNS time to live field; “class” is the standard DNS classfield (i.e., IN); “priority” is the priority of the target host (lowervalue means more preferred); “weight” is a relative weight for recordswith the same priority; “port” is the TCP or UDP port on which theservice is to be found; and “target” is the canonical hostname of themachine providing the service.

The priority field determines the precedence of use of the record'sdata. When attempting to connect with the corresponding cloud-basedservice, clients use the SRV record with the lowest-numbered priorityvalue first, and fall back to other records of equal or higher priorityif the connection to the host fails. If a service has multiple SRVrecords with the same priority value, clients use the weight field todetermine which host to use. The weight value is relevant only inrelation to other weight values for the service, and only among recordswith the same priority value. The DNS SRV is defined in RFC 2782 fromthe IETF Network Working Group published in February 2000, the entiretyof which is incorporated herein by reference for all purposes.

Some Internet protocols such as the Session Initiation Protocol (SIP)often employ or even require SRV support by network elements. SIP is anIETF-defined signaling protocol widely used for controllingcommunication sessions such as voice and video calls over the InternetProtocol (IP). SIP can be used for creating, modifying and terminatingtwo-party (unicast) or multiparty (multicast) sessions, and may be oneof the core protocols employed by systems such as shown in and describedabove with reference to FIGS. 1 and 2. The latest version of the SIPspecification is RFC 3261 from the IETF Network Working Group publishedin June 2002, the entirety of which is incorporated herein by referencefor all purposes.

According to a specific class of implementations, client devicescompatible with DNS SRV are provisioned with a DNS name associated withmultiple DNS SRV records, each of which, in this example, corresponds toone of a plurality of SIP end points (i.e., servers) associated withthat DNS name. As will be discussed, such client devices are thereforeable to locate and perform failover to alternative SIP end pointswithout rediscovery or reprovisioning. An example of such animplementation may be understood with reference to FIGS. 3 and 4.

FIG. 3 shows an example of a services system 300 (e.g., avoice/video-over-IP communication system as described above withreference to FIGS. 1 and 2) having two data centers; i.e., POP1 andPOP2. Each data center includes N SIP end points; i.e., servers 302-312.Each of Device1 and Device 2 is provisioned with one or more DNS SRVrecords for each data center, with the SRV record(s) for one data center(e.g., preferably the geographically nearer data center) being primaryfor that device and the SRV record(s) for the other data center beingsecondary for that device. In the particular implementation shown, eachdevice is provisioned with multiple SRV records for each data center,with the SRV records for the closer data center being the primary poolof SIP end points and the SRV records for the other data center beingthe secondary pool of SIP end points.

FIG. 4 shows examples of DNS SRV records for the system of FIG. 3 inwhich the number N of SIP end points in each data center is 3. As shown,the DNS SRV records for client devices closer to POP1, e.g., Device1,include records for all 6 SIP end points in both data centers, with therecords for the end points in POP1 (sip11, sip12, and sip13) having ahigher priority, i.e., in this example a priority field specifying “0,”than the priority of end points in POP2 (sip21, sip22, and sip23), i.e.,a priority field specifying “1.” Similarly, the DNS SRV records forclient devices closer to POP2, e.g., Device 2, also include records forall 6 SIP end points, but with the priorities reversed.

In this example, the weights within each priority group are the same, inwhich case the SIP end point to which the client device connects wouldbe chosen at random from among available SIP end points in a prioritygroup. This effects a load balancing among the end points in a prioritygroup which may be manipulated by adjusting the weights to achieve adesired traffic distribution among network resources. The weights mightalso be manipulated for other purposes, e.g., to effect a desired orderin which client devices request connections in cases where multipleconnection requests are required.

When a provisioned client device attempts to connect with the servicebeing provided by system 300 it first attempts to connect with an endpoint in the highest priority group of its DNS SRV records. This has thedesirable effect that the client device first attempts to connect withthe geographically closer data center. If the requested end point in thefirst priority group is not available, the client device automaticallyattempts to connect with one of the other end points in the same group.If none of the end points in the first priority group are available, theclient device then automatically attempts to connect with one of the endpoints in the second priority group, i.e., the client deviceautomatically fails over to the other data center.

It should be noted that implementations are contemplated in which thenumber of priority groups and/or the number of SRV records (andtherefore end points) within each priority group with which clientdevices may be provisioned may vary from the example described above.For example, some implementations may include additional priority groupsbeyond the two described. In addition, each priority group may includeanywhere from one to an arbitrary number of SRV records. Weighting withand between or among priority groups may also be manipulated to achievea variety of effects. Other variations will be apparent to those ofskill in the art.

FIG. 5 illustrates an example of how a client device may be provisionedand register with a service system such as the system of FIGS. 3 and 4.Client device 502 sends a configuration request to registration (RGS)server 504 which determines the client device type and sends aconfiguration response back to the client device with the appropriateDNS name. Client device 502 then transmits a service request to anappropriate DNS server 506 which responds with a set of SRV records forthe corresponding service. In this example, DNS server 506 returns theSRV records corresponding to POP2 and POP1 in the example of FIGS. 3 and4, with the records for POP2 being higher priority. Client device 502then selects one of the SRV records from the higher priority group (inthis case the one corresponding to end point sip22 and generates aconnection request for that end point. DNS server 506 replies with theIP address, with which client device 502 generates a registrationrequest to the corresponding SIP end point 508. As discussed above,because it's been provisioned with multiple SRV records for multipledata centers, client device 502 can automatically attempt to connectwith other end points if an attempt to connect fails for any reason.

As will be understood by those of skill in the art, not all clientdevices which attempt to connect with a services system (such as thosedescribed above with reference to FIGS. 1-3) support advanced DNSresolving mechanisms such as DNS SRV. A particular challenge insupporting such devices relates to the fact that many failover schemesfor cloud-based services require the client device to discover the newlocation to reconnect with the service. There are a number of variableson the client side that make such schemes inefficient. Even when donerelatively gracefully, a conventional DNS failover can take minutes.This is clearly unacceptable for many services.

As with the DNS SRV failover mechanism described above, the idea is toprovide a failover mechanism such that, upon a failure or interruptionoccurring, the service remains available to any client device (includingthose that do not support DNS SRV) without the client device being awarethat a change has occurred. Therefore, according to another class ofimplementations, high availability is promoted for such devices using IPanycast.

IP anycast (also referred to simply as “anycast”) is a networkaddressing and routing methodology in which datagrams from a singlesender are routed to the topologically nearest node in a group ofpotential receivers all identified by the same destination address. Onthe Internet, anycast is typically implemented by using the BorderGateway Protocol (BGP) to simultaneously announce the same destinationIP address range from multiple different places on the Internet. Thisresults in packets addressed to destination addresses in this rangebeing routed to the geographically nearest point on the Internetannouncing the given destination IP address. The use of anycast toprovide DNS services is described in RFC 3258 from the IETF NetworkWorking Group published in April 2002, the entirety of which isincorporated herein by reference for all purposes.

BGP is the protocol which makes core routing decisions on the Internet.It maintains a table of IP networks or prefixes which designate network“reach-ability” among autonomous systems. It is often described as apath vector protocol. When BGP runs between two peers in the sameautonomous system it is referred to as Internal BGP (IBGP). When it runsbetween autonomous systems it is referred to as External BGP (EBGP).Version 4 of BGP is described in RFC 4271 from the IETF Network WorkingGroup published in January 2006, the entirety of which is incorporatedherein by reference for all purposes.

Referring now to the example illustrated in FIG. 6, an IP address rangeA.B.C.# is advertised for both data centers 602 and 604 of a servicessystem 600. Client devices near one or the other location and attemptingto connect using address in the advertised range will be more likely toconnect with the geographically closer location according to BGP. Insome cases, both data centers can be “live” and responsible forproviding the service to the client devices that connect with them dueto geographic proximity. In other cases where only one of the locationsis live, the traffic that tends toward the dormant location may berouted internally to the live location, e.g., using IBGP or the OpenShortest Path First (OSPF) protocol, via a high bandwidth, cross-POPlink 606. This can be done, for example, by statically configuring edgerouters at the data center or dynamically configuring edge routers usinga suitable route health injection mechanism.

OSPF is an adaptive routing protocol for Internet Protocol (IP)networks. OSPF is an interior gateway protocol that routes InternetProtocol (IP) packets within a single routing domain (e.g., anautonomous system). It gathers link state information from availablerouters and constructs a topology map of the network. The topologydetermines the routing table presented to the Internet Layer which makesrouting decisions based on the destination IP address found in IPpackets. OSPF is described in RFC 2328 from the IETF Network WorkingGroup published in April 1998, and RFC 5340 from the IETF NetworkWorking Group published in July 2008, the entirety of each of which isincorporated herein by reference for all purposes.

In either case, if a failure or interruption of some kind occurs at oneof the data centers, servicing of the clients being handled by thefailed location may be routed internally to the other location, e.g.,via cross-POP link 606. If the failure results in an inability toconnect with the failed data center entirely, the devices being handledby that location can connect with the other data center using the sameIP address. Advertisement of the IP address range at the failed locationmay also be terminated, in which case the client devices being handledby that location will connect with the other data center directly.Therefore, even though some clients may experience an interruption inservice, reconnection to the service may be done using the same IPaddress, thereby minimizing the interruption.

According to a particular implementation, if the system detects that asingle client device attempts (e.g., smart phone 608) to connect withmore than one of the data centers advertising the same IP address rangewithin a relatively short programmable period of time, the system canswitch that client device to a unicast approach. That is, the systemprovisions that client device with a unique address for only one of thedata centers for accessing the service. This addresses the possibleinstability that may result for some client devices in the anycastscenario, e.g., where the client device is equidistant to two datacenters.

In another example of the use of anycast illustrated in FIG. 7, IPaddress ranges A.B.C.# and X.Y.Z.# are both advertised for both datacenters 702 and 704 of services system 700, both of which are intendedto be active. In the depicted example, the range A.B.C.# is the primaryrange for data center 702 and the range X.Y.Z.# is the primary range fordata center 704. As with the previous example described above withreference to FIG. 6, client devices will tend to connect with thegeographically closer data center. However, as shown in this example, ifa client device connects with the data center using an IP address in theprimary range of the other data center, that connection is routedinternally to the other data center for handling, e.g., via a cross-POPlink 706. Again, this may be handled by static or dynamic configurationof edge routers.

If either of the data centers goes down, all traffic for both IP addressranges will, by default, automatically go to the other data center. Ifthe failure or interruption at one of the data centers is lesscatastrophic, e.g., the data center can still receive traffic but aninsufficient number of SIP end points are available, some or all of thetraffic that would normally be handled locally can be internally routedto the other data center, e.g., via a cross-POP link. If the cross-POPlink goes down, both data centers may be configured to handle trafficfor both address ranges locally.

According to some implementations, the handling of failover scenarioscan be achieved by manual reconfiguration of network resources.According to other implementations, route health injection mechanismsmay be employed to enable the dynamic configuration of network resourcesto handle the various failover scenarios. Each data center would haveone or more health checkers which monitor the health of the variouscomponents of the system and, when a failure is detected, sendinstructions (or alert another device to send instructions) toappropriate system components to appropriately reconfigure the system.For example, one or more health checkers placed in a first data centercan be configured to monitor system components in a second data centerso that when the second data center encounters problems, the system canstill be reconfigured by the health checkers placed in the first datacenter. In another example, one or more health checkers can be placedoutside of the data centers to monitor the health of various systemcomponents in the data centers and send instructions to reconfigure thesystem as appropriate.

For example, if it is determined that an insufficient number of SIP endpoints are available in a data center, the routing tables of an edgerouter associated with the data center may be dynamically reconfiguredto switch the traffic to another data center. In another example inwhich the same address ranges are associated with multiple data centers,if it is determined that another data center for which a particularaddress range had been designated as the primary range is no longeraccessible, the routing tables of an edge router may be dynamicallyreconfigured to allow local handling of traffic directed to the primaryaddress range of the failed data center rather than rerouting it. In yetanother example, if it is determined that SIP end points correspondingto particular IP addresses in the range associated with a data centerare not available, those IP addresses may be reassigned to other endpoints associated with that data center. Other examples of the handlingof a wide variety of failover scenarios will be apparent to those ofskill in the art.

According to another class of implementations, DNS SRV is employed topromote high availability of a services system (e.g., a communicationssystem as described above) in a manner similar to that described abovewith reference to FIGS. 3-5 except that one or more of the SRV recordswith which a client device is provisioned corresponds to an anycastaddress that may be associated with one or more SIP end points in one ormore data centers of the system. FIG. 8 shows an example of a set of SRVrecords with which a client might be provisioned in which the first sixrecords correspond to end points in two different data centers in amanner similar to the sets of 6 shown in and described above withreference to FIG. 4. By contrast, the seventh and eighth SRV records inthis example correspond to anycast addresses. As will become clear, suchimplementations provide failover mechanisms on both the client side andat the data center that govern how to reconnect with the service in theevent of a failure rather than relying on a single mechanism.

When a client device that supports DNS SRV registers with the servicessystem it is provisioned with a set of SRV records as discussed abovewith reference to FIG. 5. However, in this case, at least one (andpossibly more than one) of the SRV records corresponds to an anycastaddress. According to some implementations, the SRV record(s)corresponding to the anycast address(es) are lower in priority than theone or more SRV records corresponding to unicast addresses. For example,the SRV records might include a first SRV record having the highestpriority which corresponds to a unicast address associated with thegeographically closest data center; a second SRV record having a lowerpriority which corresponds to a unicast address associated with ageographically more distant data center; and an anycast addressassociated with both data centers.

When attempting to access the service in this example scenario, theclient device provisioned according to these SRV records first attemptsto connect with the unicast address specified by the highest prioritySRV record. If the attempt is unsuccessful, or if the corresponding endpoint or data center goes down after a connection is established, theclient device automatically attempts to connect with the unicast addressspecified by the second, lower priority SRV record. Again, if theattempt is unsuccessful or the connection is interrupted, the clientdevice proceeds to the next option which, in this case, is the anycastaddress specified by the third SRV record. This connection may beestablished and/or failed over as described above with reference to theFIGS. 6 and 7.

It should be noted that implementations employing various combinationsof the high availability mechanisms described herein are contemplated.For example, implementations of services systems are contemplated whichsupport some client devices with DNS SRV, and other client devices withsingle-address anycast. Implementations of services systems are alsocontemplated which support some client devices with DNS SRV in whichsome SRV records correspond to anycast addresses, and other clientdevices with single-address anycast. Other suitable variations of thefunctionalities and techniques described herein will be apparent tothose of skill in the art.

According to some implementations, the priority and/or weighting of theSRV records with which a client device is provisioned may vary dependingon a variety of parameters, e.g., the geographic location of therequesting client device, or the capacity and/or load of particular datacenters. For example, in a system in which multiple data centersadvertise the same anycast address ranges (e.g., as discussed above withreference to FIG. 7), a client device might have multiple SRV recordscorresponding to anycast addresses in multiple ranges that areprioritized to favor the primary range(s) of the geographically closerdata center(s).

It should be noted that, despite references to particular computingparadigms and software tools herein, the computer program instructionswith which embodiments of the invention may be implemented maycorrespond to any of a wide variety of programming languages andsoftware tools, and be stored in any type of volatile or nonvolatile,non-transitory computer-readable storage medium or memory device, andmay be executed according to a variety of computing models including,for example, a client/server model, a peer-to-peer model, on astand-alone computing device, or according to a distributed computingmodel in which various of the functionalities described herein may beeffected or employed at different locations. In addition, reference toparticular protocols herein are merely by way of example. Suitablealternatives known to those of skill in the art may be employed withoutdeparting from the scope of the invention.

While the invention has been particularly shown and described withreference to specific embodiments thereof, it will be understood bythose skilled in the art that changes in the form and details of thedisclosed embodiments may be made without departing from the spirit orscope of the invention. For example, although various embodimentsdescribed herein refer to SIP as the application layer protocol, andalthough there are advantages associated with the use of SIP (e.g.,supports multiple transport protocols (e.g., TCP, UDP, TCP+TLS), supportfor IPv4 and IPv6 is inheritable, etc.), services employing otherapplication layer protocols may be implemented to take advantage of theautomatic failover mechanisms described.

In another example, in implementations using anycast, an IP address thatis in the address range associated with a particular data center maycorrespond to one or more SIP end points in that data center. Havingmultiple end points correspond to one IP address might be useful, forexample, to support load balancing among the end points using any of avariety of load balancing techniques.

In another example, although implementations employing anycast have beendescribed above with examples showing two data centers and one or twoaddress ranges, it will be understood that implementations arecontemplated in which an arbitrary number of data centers and addressranges are employed.

Finally, although various advantages, aspects, and objects of thepresent invention have been discussed herein with reference to variousembodiments, it will be understood that the scope of the inventionshould not be limited by reference to such advantages, aspects, andobjects. Rather, the scope of the invention should be determined withreference to the appended claims.

What is claimed is:
 1. A computer-implemented method for accessing aservice provided by a services network using a client device, theservice having a plurality of service end points associated with aplurality of geographically distinct data centers, the methodcomprising: retrieving a first one of a plurality of domain name system(DNS) service records that are simultaneously stored by the clientdevice, each of a first subset of the DNS service records correspondingto a unicast destination address, each unicast destination address beingassociated with one or more of the service end points in only a singleone of the data centers, and each of a second subset of the DNS servicerecords corresponding to an anycast destination address, each anycastdestination address being simultaneously associated with one or more ofthe service end points in at least two of the data centers; transmittinga first request to access the service to the destination addresscorresponding to the first DNS service record; where the first requestto access the service is not successful, or where a connection to theservice established using the first DNS service record is interrupted,retrieving a second one of the plurality of DNS service records, thesecond DNS service record being included in the second subset of DNSservice records; and transmitting a second request to access the serviceto the destination address corresponding to the second DNS servicerecord.
 2. The method of claim 1, wherein the first subset of DNSservice records is higher priority than the second subset of DNS servicerecords, and wherein attempts to connect with the service are made usingall of the first subset of DNS service records before an attempt toconnect with the service is made using any of the second subset of DNSservice records.
 3. The method of claim 1, wherein the first subset ofDNS service records includes a first group of one or more DNS servicerecords corresponding to a first one of the data centers, and a secondgroup of one or more DNS service records corresponding to a second oneof the data centers, and wherein the first group of DNS service recordsis higher priority than the second group of DNS service records, andwherein attempts to connect with the service are made using all of thefirst group of DNS service records before an attempt to connect with theservice is made using any of the second group of DNS service records. 4.The method of claim 1, wherein the second subset of DNS service recordsincludes a first group of one or more DNS service records correspondingto both of first and second ones of the data centers, and a second groupof one or more DNS service records corresponding to both of the firstand second data centers, and wherein the first group of DNS servicerecords is higher priority than the second group of DNS service records,and wherein attempts to connect with the service are made using all ofthe first group of DNS service records before an attempt to connect withthe service is made using any of the second group of DNS servicerecords.
 5. The method of claim 1, wherein the first subset of DNSservice records includes two or more DNS service records each having apriority associated therewith, the priorities of the two or more DNSservice records in the first subset of DNS services recordscorresponding to a load balancing scheme of the service.
 6. The methodof claim 1, wherein the service comprises a voice-over-IP service. 7.The method of claim 1, wherein the client device comprises one of a cellphone, a smart phone, a tablet, a laptop computer, a desktop computer, atelephone, an Internet Protocol phone, a set top box, or a gamingconsole.
 8. A client device configured to access a service provided by aservices network having a plurality of service end points associatedwith a plurality of geographically distinct data centers, the clientdevice comprising one or more processors and memory configured to accessthe service using a plurality of domain name system (DNS) servicerecords that are simultaneously stored in the memory of the clientdevice, each of a first subset of the DNS service records correspondingto a unicast destination address, each unicast destination address beingassociated with one or more of the service end points in only a singleone of the data centers, and each of a second subset of the DNS servicerecords corresponding to an anycast destination address, each anycastdestination address being simultaneously associated with one or more ofthe service end points in at least two of the data centers, each of theDNS service records being configured to enable access to the serviceusing the corresponding destination address.
 9. The client device ofclaim 8, wherein the one or more processors and memory of the clientdevice are configured to, where a first request to access the serviceusing a first one of the DNS service records is not successful, or wherea connection to the service established using the first DNS servicerecord is interrupted, request access to the service using a second oneof the plurality of DNS service records, the second DNS service recordbeing included in the second subset of DNS service records.
 10. Theclient device of claim 8, wherein the first subset of DNS servicerecords is higher priority than the second subset of DNS servicerecords, and wherein the one or more processors and memory of the clientdevice are configured to attempt to connect with the service using allof the first subset of DNS service records before attempting to connectwith the service using any of the second subset of DNS service records.11. The client device of claim 8, wherein the first subset of DNSservice records includes a first group of one or more DNS servicerecords corresponding to a first one of the data centers, and a secondgroup of one or more DNS service records corresponding to a second oneof the data centers, and wherein the first group of DNS service recordsis higher priority than the second group of DNS service records, andwherein the one or more processors and memory of the client device areconfigured to attempt to connect with the service using all of the firstgroup of DNS service records before attempting to connect with theservice using any of the second group of DNS service records.
 12. Theclient device of claim 8, wherein the second subset of DNS servicerecords includes a first group of one or more DNS service recordscorresponding to both of first and second ones of the data centers, anda second group of one or more DNS service records corresponding to bothof the first and second data centers, and wherein the first group of DNSservice records is higher priority than the second group of DNS servicerecords, and wherein the one or more processors and memory of the clientdevice are configured to attempt to connect with the service using allof the first group of DNS service records before attempting to connectwith the service using any of the second group of DNS service records.13. The client device of claim 8, wherein the first subset of DNSservice records includes two or more DNS service records each having apriority associated therewith, the priorities of the two or more DNSservice records in the first subset of DNS services recordscorresponding to a load balancing scheme of the service.
 14. The clientdevice of claim 8, wherein the client device comprises one of a cellphone, a smart phone, a tablet, a laptop computer, a desktop computer, atelephone, an Internet Protocol phone, a set top box, or a gamingconsole.